Simple compositing data packager for freelancers #884

jakubjezek001 · 2021-01-14T14:59:23Z

Details

This is kick of POC for packaging tool. This is Ftrack Event server action. At the moment it is able to find last nuke workfile (.nk) parse any available paths from the nuke script which is practically text file and pack the data into zip file.

All paths are rewritten back to the file with Env Var for a project root. So artist can drop the data enywhere at his machine and set the Env Var to his own root. After his work is done he only needs to send the nuke workfile back to the studio and they can render all without path remapping.

If any of Read nodes with linked sources outside of project are in nuke script and event server is having linked all required storages then all are relinked to [nuke workfile root]/resources/ during the processing.

Potentially to future of this tool is to be able do this for any task and host.

Testing notes

add following code to anatomy template pype-config/anatomy/default.yaml or to any of your project's overrides.

delivery:
    packaging: "{root}/{project[name]}/outsorcing-workload/{asset}-{task}.{ext}"

Dependencies:

ynput/pype-setup#84

…load_package_for_tasks` function

…kaging

…ackage

hound

Some files could not be reviewed due to errors:

Traceback (most recent call last):

Traceback (most recent call last):
  File "/home/linters/.local/bin/flake8", line 11, in 
    sys.exit(main())
  File "/home/linters/.local/lib/python3.6/site-packages/flake8/main/cli.py", line 18, in main
    app.run(argv)
  File "/home/linters/.local/lib/python3.6/site-packages/flake8/main/application.py", line 393, in run
    self._run(argv)
  File "/home/linters/.local/lib/python3.6/site-packages/flake8/main/application.py", line 381, in _run
    self.run_checks()
  File "/home/linters/.local/lib/python3.6/site-packages/flake8/main/application.py", line 300, in run_checks
    self.file_checker_manager.run()
  File "/home/linters/.local/lib/python3.6/site-packages/flake8/checker.py", line 331, in run
    self.run_serial()
  File "/home/linters/.local/lib/python3.6/site-packages/flake8/checker.py", line 315, in run_serial
    checker.run_checks()
  File "/home/linters/.local/lib/python3.6/site-packages/flake8/checker.py", line 598, in run_checks
    self.run_ast_checks()
  File "/home/linters/.local/lib/python3.6/site-packages/flake8/checker.py", line 502, in run_ast_checks
    for (line_number, offset, text, check) in runner:
  File "/home/linters/.local/lib/python3.6/site-packages/flake8_django/checker.py", line 56, in run
    parser.visit(self.tree)
  File "/usr/lib/python3.6/ast.py", line 253, in visit
    return visitor(node)
  File "/usr/lib/python3.6/ast.py", line 261, in generic_visit
    self.visit(item)
  File "/home/linters/.local/lib/python3.6/site-packages/flake8_django/checker.py", line 39, in visit_ClassDef
    self.capture_issues_visitor('ClassDef', node)
  File "/home/linters/.local/lib/python3.6/site-packages/flake8_django/checker.py", line 33, in capture_issues_visitor
    self.generic_visit(node)
  File "/usr/lib/python3.6/ast.py", line 263, in generic_visit
    self.visit(value)
  File "/home/linters/.local/lib/python3.6/site-packages/flake8_django/checker.py", line 36, in visit_Call
    self.capture_issues_visitor('Call', node)
  File "/home/linters/.local/lib/python3.6/site-packages/flake8_django/checker.py", line 30, in capture_issues_visitor
    issues = checker.run(node)
  File "/home/linters/.local/lib/python3.6/site-packages/flake8_django/checkers/render.py", line 22, in run
    if isinstance(arg, ast.Call) and arg.func.id == 'locals':
AttributeError: 'Attribute' object has no attribute 'id'

hound

Some files could not be reviewed due to errors:

Traceback (most recent call last):

Traceback (most recent call last):
  File "/home/linters/.local/bin/flake8", line 11, in 
    sys.exit(main())
  File "/home/linters/.local/lib/python3.6/site-packages/flake8/main/cli.py", line 18, in main
    app.run(argv)
  File "/home/linters/.local/lib/python3.6/site-packages/flake8/main/application.py", line 393, in run
    self._run(argv)
  File "/home/linters/.local/lib/python3.6/site-packages/flake8/main/application.py", line 381, in _run
    self.run_checks()
  File "/home/linters/.local/lib/python3.6/site-packages/flake8/main/application.py", line 300, in run_checks
    self.file_checker_manager.run()
  File "/home/linters/.local/lib/python3.6/site-packages/flake8/checker.py", line 331, in run
    self.run_serial()
  File "/home/linters/.local/lib/python3.6/site-packages/flake8/checker.py", line 315, in run_serial
    checker.run_checks()
  File "/home/linters/.local/lib/python3.6/site-packages/flake8/checker.py", line 598, in run_checks
    self.run_ast_checks()
  File "/home/linters/.local/lib/python3.6/site-packages/flake8/checker.py", line 502, in run_ast_checks
    for (line_number, offset, text, check) in runner:
  File "/home/linters/.local/lib/python3.6/site-packages/flake8_django/checker.py", line 56, in run
    parser.visit(self.tree)
  File "/usr/lib/python3.6/ast.py", line 253, in visit
    return visitor(node)
  File "/usr/lib/python3.6/ast.py", line 261, in generic_visit
    self.visit(item)
  File "/home/linters/.local/lib/python3.6/site-packages/flake8_django/checker.py", line 39, in visit_ClassDef
    self.capture_issues_visitor('ClassDef', node)
  File "/home/linters/.local/lib/python3.6/site-packages/flake8_django/checker.py", line 33, in capture_issues_visitor
    self.generic_visit(node)
  File "/usr/lib/python3.6/ast.py", line 263, in generic_visit
    self.visit(value)
  File "/home/linters/.local/lib/python3.6/site-packages/flake8_django/checker.py", line 36, in visit_Call
    self.capture_issues_visitor('Call', node)
  File "/home/linters/.local/lib/python3.6/site-packages/flake8_django/checker.py", line 30, in capture_issues_visitor
    issues = checker.run(node)
  File "/home/linters/.local/lib/python3.6/site-packages/flake8_django/checkers/render.py", line 22, in run
    if isinstance(arg, ast.Call) and arg.func.id == 'locals':
AttributeError: 'Attribute' object has no attribute 'id'

hound

Some files could not be reviewed due to errors:

Traceback (most recent call last):

Traceback (most recent call last):
  File "/home/linters/.local/bin/flake8", line 11, in 
    sys.exit(main())
  File "/home/linters/.local/lib/python3.6/site-packages/flake8/main/cli.py", line 18, in main
    app.run(argv)
  File "/home/linters/.local/lib/python3.6/site-packages/flake8/main/application.py", line 393, in run
    self._run(argv)
  File "/home/linters/.local/lib/python3.6/site-packages/flake8/main/application.py", line 381, in _run
    self.run_checks()
  File "/home/linters/.local/lib/python3.6/site-packages/flake8/main/application.py", line 300, in run_checks
    self.file_checker_manager.run()
  File "/home/linters/.local/lib/python3.6/site-packages/flake8/checker.py", line 331, in run
    self.run_serial()
  File "/home/linters/.local/lib/python3.6/site-packages/flake8/checker.py", line 315, in run_serial
    checker.run_checks()
  File "/home/linters/.local/lib/python3.6/site-packages/flake8/checker.py", line 598, in run_checks
    self.run_ast_checks()
  File "/home/linters/.local/lib/python3.6/site-packages/flake8/checker.py", line 502, in run_ast_checks
    for (line_number, offset, text, check) in runner:
  File "/home/linters/.local/lib/python3.6/site-packages/flake8_django/checker.py", line 56, in run
    parser.visit(self.tree)
  File "/usr/lib/python3.6/ast.py", line 253, in visit
    return visitor(node)
  File "/usr/lib/python3.6/ast.py", line 261, in generic_visit
    self.visit(item)
  File "/home/linters/.local/lib/python3.6/site-packages/flake8_django/checker.py", line 39, in visit_ClassDef
    self.capture_issues_visitor('ClassDef', node)
  File "/home/linters/.local/lib/python3.6/site-packages/flake8_django/checker.py", line 33, in capture_issues_visitor
    self.generic_visit(node)
  File "/usr/lib/python3.6/ast.py", line 263, in generic_visit
    self.visit(value)
  File "/home/linters/.local/lib/python3.6/site-packages/flake8_django/checker.py", line 36, in visit_Call
    self.capture_issues_visitor('Call', node)
  File "/home/linters/.local/lib/python3.6/site-packages/flake8_django/checker.py", line 30, in capture_issues_visitor
    issues = checker.run(node)
  File "/home/linters/.local/lib/python3.6/site-packages/flake8_django/checkers/render.py", line 22, in run
    if isinstance(arg, ast.Call) and arg.func.id == 'locals':
AttributeError: 'Attribute' object has no attribute 'id'

mkolar

works really nice and fill the immediate need. We'll keep expanding on this after a larger brainstorm.

jakubjezek001 and others added 22 commits January 12, 2021 13:51

feat(nuke): encoding env var in filepath knob

5c06f10

feat(global, ftrack): event for packaging jobs for artists

dcb0a56

feat(nuke): clean obsolete code

7bca7fb

don't use lib.packaging as singleton

1a391c3

save some indentation

27f38d0

renamed event to action

2bc1e5e

make_workload_package has different attributes and added `make_work…

d0da488

…load_package_for_tasks` function

added prepare_data function that collects all required data for pac…

da6fff6

…kaging

implemented make_workfload_package_for_tasks

2034d0e

moved AvalonMongoDB objec creation to __init__

82c8aa2

allowed_task_names is class attribute

bba1971

added proper discover filtering

7156081

don't raise an exception but return message with information

814a140

implemented find_asset_docs_by_name fallback

4f2f91a

implemented collecting of entities info for packaging and executing p…

2aa3aba

…ackage

added some docstring to action

76fda6b

small tweaks

c74c785

do not double close file stream

d706f1c

make sure package directory exists

a5cb2ba

fix last workfile path

37865fc

fix dbcon in action

068eb00

allow multiroot setup

a967768

jakubjezek001 added type: enhancement Enhancements to existing functionality 2x-dev labels Jan 14, 2021

jakubjezek001 requested a review from a team January 14, 2021 14:59

jakubjezek001 assigned jakubjezek001 and iLLiCiTiT Jan 14, 2021

hound bot reviewed Jan 14, 2021

View reviewed changes

hound(global): improvements

9b88040

hound bot reviewed Jan 14, 2021

View reviewed changes

jakubjezek001 linked an issue Jan 14, 2021 that may be closed by this pull request

Collect shot data for external artist #878

Closed

3 tasks

feat(global): removing temp packaging dir after zip is created

e49f376

hound bot reviewed Jan 14, 2021

View reviewed changes

mkolar added this to the 2.15.0 milestone Jan 20, 2021

mkolar approved these changes Jan 20, 2021

View reviewed changes

mkolar merged commit 47217a9 into 2.x/develop Jan 20, 2021

mkolar deleted the feature/2x-878-collect-shot-data-for-external-artist branch January 26, 2021 08:49

mkolar changed the title ~~Packaging Task for outsorcing~~ Simple compositing data packager for freelancers Jan 27, 2021

nextreleaseio bot mentioned this pull request Feb 8, 2021

2.15.0 #996

Closed

mkolar mentioned this pull request Sep 8, 2021

Collect shot data for external artist #878

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Simple compositing data packager for freelancers #884

Simple compositing data packager for freelancers #884

jakubjezek001 commented Jan 14, 2021 •

edited

Loading

hound bot left a comment

hound bot left a comment

hound bot left a comment

mkolar left a comment

Simple compositing data packager for freelancers #884

Simple compositing data packager for freelancers #884

Conversation

jakubjezek001 commented Jan 14, 2021 • edited Loading

Details

Testing notes

Dependencies:

hound bot left a comment

Choose a reason for hiding this comment

hound bot left a comment

Choose a reason for hiding this comment

hound bot left a comment

Choose a reason for hiding this comment

mkolar left a comment

Choose a reason for hiding this comment

jakubjezek001 commented Jan 14, 2021 •

edited

Loading